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Abstract 

This  paper  presents  the  PIC  matrices,  a  computationally  efficient  subclass  of  link  matrices  that  may  be 
considered  for  the  interpretation  of  query  operators  in  the  INQUERY  inference  network.  The  PIC  class  of 
matrices  is  formally  defined;  a  0(n2)  algorithm,  PIC-EVAL,  is  presented  for  the  evaluation  of  members  of 
this  class;  and  a  proof  of  the  correctness  of  the  algorithm  is  given.  A  further  specialization  of  this  class  that 
can  be  evaluated  even  more  efficiently  is  discussed.  Finally,  a  generalization  of  the  PIC  class,  for  which 
the  input  probabilities  may  be  viewed  as  weighted,  is  defined,  and  a  simple  modification  of  the  PIC-EVAL 
algorithm  that  allows  for  the  evaluation  of  matrices  of  this  extended  class  is  given. 


1  Introduction 

The  search  engine  of  the  INQUERY  system  is  based  on  the  use  of  inference  networks  as  developed  in  Howard 
Turtle’s  doctoral  dissertation.  This  document  presents  ideas  for  the  efficient  evaluation  of  link  matrices.  We 
begin  with  a  brief  description  of  how  inference  networks  are  used  in  INQUERY  and  the  problems  of  efficient 
computation  that  present  themselves.  We  then  introduce  the  notation  that  will  be  adopted  for  this  paper  and, 
using  it,  give  a  formal  definition  of  PIC  matrices,  the  subclass  of  link  matrices  that  will  be  the  focus  of  our 
attention. 


1.1  link  matrices  in  inquery 

The  retrieval  mechanism  in  the  INQUERY  system  is  modeled  as  an  inference  network.  An  inference  network  is 
a  directed  acyclic  graph  that  is  used  to  represent  a  discrete  joint  probability  distribution,  each  node  associated 
with  a  distinct  variable. 

The  topology  of  an  inference  network  is  interpreted  as  encoding  a  set  of  conditional  independence  conditions. 
If  the  nodes  corresponding  to  the  variables.  Pi, ,  Pn,  are  the  immediate  predecessors  ( parents )  of  a  node, 
Q,  and  Z\, ...,  Zs  are  all  other  nodes  that  are  not  descendents  of  (i.e.  are  not  reachable  from)  Q,  as  shown  in 
figure  1,  then  Q  is  considered  to  be  conditionally  independent  of  Zt . ....  Zs  given  Pi, ,  Pn.  That  is: 

pr(Q  is  true  |  P1} . . .  ,Pn,  Zx, . . . ,  Zs)  =  pr(Q  is  true  |  P1,...,Pn) 
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Figure  1 :  part  of  an  inference  net 


Given  probabilities  for  the  root  nodes  (i.e.  nodes  with  no  parents),  the  network  may  be  processed  in  a 
top-down  fashion  in  order  to  produce  the  probabilities  relevant  to  each  of  its  nodes.  As  a  consequence  of 
the  independence  assumptions  implicit  in  the  network  topology,  once  probabilities,  p i, . . . , pn,  have  been 
produced  for  the  parents  of  a  node,  Q ,  the  probability  that  Q  takes  the  value  y  €  Dq  is  given  by: 

pr(Q  =  y)=  ^2pr(Q  =y  \  P1=x1,...,Pn  =  xn)pr(Pi  =  x1  )---pr(Pn  =  xn) 

»l£Di . ■•£»» 

where  D i, . . . ,  Dn.  Dq  are  the  sets  of  values  that  may  be  assumed  by  the  variables.  Pi, ... ,  P„,  and  Q. 
respectively. 

In  INQUERY  each  node  corresponds  to  a  proposition;  that  is,  a  variable  that  may  take  on  one  of  two  values: 
true  or  false.  For  example,  in  figure  1,  each  P,  might  correspond  to  the  proposition  that  some  document 
under  consideration  is  ” about”  some  concept,  Ci,  while  Q  corresponds  to  the  proposition  that  the  document 
is  relevant. 

Since  all  the  variables  are  binary  valued  in  INQUERY,  the  dependence  of  a  child  on  its  parents  can  be  given 
via  the  specification  of: 

pr(Q  is  true  |  Pi  =&!,.  ...  P„  =  bn)  and 
pr(Q  is  false  |  Px  =  bi, . . . ,  Pn  =  bn) 

for  each: 

<  bi , . . . ,  bn  >  €  {true,  false}" 

Equivalently,  the  values: 

olr  =  pr{Q  is  false  |  i  €  R  =>  Pi  is  true 

i  R  ^  Pi  is  false)  and 

aR  =  pr(Q  is  true  |  i  €  R  =>•  Pj  is  true 

i  (f:  R  =>•  Pi  is  false) 

must  be  specified  for  every  possible  subset,  R,  of  {1, . . . ,  n}.  In  terms  of  these  conditional  properties,  the 
probability  that  a  child  node  is  true  can  be  calculated  once  the  probabilities  of  truth,  pi, . . . , pn.  are  known 
for  each  of  its  parent  nodes: 

pr(Q  is  false)  =  E  OtR  IMa  -  Pi)  and 

HC{1 . n}  i£R  i$R 

pr(Q  is  true)  =  ^  aR^p^il  -  pf) 

»C{l,..,n)  iefl  iiR 


The  set  of  coefficients  involved  is  conveniently  organized  as  a  2  x  2"  matrix: 
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-Po. ..000 
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-P0...010 

Pi... 111 

Q  false 

(*0...000 

c*o...ooi 

(to.. .010 

Oil. ..Ill 

Q  true 

ao.. .000 

QU..001 

(to. ..010 

ctl. ..Ill 

where  ab1,b2,...b„  is  the  probability  that  Q  is  true  subject  to  the  condition  that  the  parents  P,  such  that  b;  =  1 
are  true,  and  the  parents  P$  such  that  b;  =  0  are  false;  dtbi,b2,...b„  is  the  corresponding  probability  that  Q  is 
false  and  is  equal  to  1  —  ain  ,b2....bri  ■  This  matrix,  known  as  a  link  matrix ,  may  be  visualized  as  linking  the 
child  node  with  the  parent  nodes  as  is  shown  in  figure  2  1 . 

(  PI  )  (  P2  )  (  Pn  ) 

\IV 


f 


Figure  2:  link  matrix  links  child  to  parents 

INQUERY  examines  documents  one  by  one.  For  each,  the  inference  network  is  used  to  evaluate  evidence 
that  the  document  satisfies  an  information  need  expressed  by  the  user.  A  given  link  matrix  form  can  be 
viewed  as  defining  an  operator  for  combining  evidence.  For  example,  suppose  the  propositions  Pi,  P2,  P3, 
state  that  three  queries  (/1 ,  (fe ;  93 .  respectively,  have  been  (in  some  sense)  satisfied  by  the  document  currently 
under  scrutiny.  A  link  matrix  connecting  the  parents  nodes.  Pi,  P2,  P3,  with  the  child  node,  Q,  can  be 
viewed  as  a  way  of  forming  a  query,  q ,  that  is  a  composite  of  the  individual  sub-queries.  The  child  node,  Q, 
would  correspond  to  the  proposition  that  the  combined  query,  q ,  has  been  satisfied.  The  specification  of  the 
coefficients  of  the  link  matrix  defines  the  way  the  sub-queries  are  combined  in  that  it  gives  all  the  information 
necessary  for  determining  the  probability  that  q  has  been  satisfied  given  the  probabilities,  p  \ ,  ju ,  ps ,  that  the 
individual  subqueries  have  been  satisfied. 

One,  admittedly  arbitrary,  way  of  defining  a  3-ary  query  composition  operator  for  forming  the  query,  q ,  from 
the  sub-queries  91 , 92,  93.  might  be  to  specify  that  q  is  to  be  considered  satisfied  with: 

•  80%  probability  if  qi  is  satisfied,  independent  of  the  whether  or  not  92  and  <73  are  satisfied; 

•  50%  probability  if  q\  is  not  satisfied,  but  q-2  and  <73  are  both  satisfied 

•  10%  probability  in  any  other  situation. 

This  particular  operator  corresponds  to  the  link  matrix: 


Tooo 

-P001 

P010 

Ton 

P100 

Pi  ot 

Tno 

Pm 

.1 

.1 

.1 

.5 

.8 

.8 

.8 

.8 

Where  the  column  under  Pon,  for  example,  gives  the  probability  that  Q  is  true  given  that  Pi  is  false,  P2 
is  true,  and  P3  is  true.  Clearly,  this  link  matrix  has  the  desired  effect.  Typically,  none  of  the  parents  will  be 
known  to  be  either  true  or  false  with  certainty.  Rather,  evidence  corresponding  to  each  of  Pi,  P2,  P3  will, 
in  general,  be  estimated  to  be  present  with  certain  probabilities:  P1.P2.P3-  Given  these  probabilities,  the 
probability  that  Q  is  true  can  be  calculated  as: 


pr(Q  is  true)  =  .1  •  P1P2P3  +  -1  ■  P1P2P3  +  -1  ■  P1P2P3  +  -5  ■  P1P2P3 

+  -8  •  P1P2P3  +  -8  •  P1P2P3  +  -8  •  P1P2P3  +  -8  •  P1P2P3 

1  Since  the  columns  of  a  link  matrix  must  sum  to  one,  from  this  point  on  only  the  row  corresponding  to  Q  true  will  be  shown. 
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1.2  efficiency  of  link  matrix  evaluation 

Each  distinct  assignment  of  values  to  the  coefficients  of  a  link  matrix  defines  a  different  operator.  This 
provides  a  large  class  of  operators  to  choose  from  when  determining,  for  example,  what  operators  might  be 
made  available  to  users  for  the  composition  of  queries.  Two  problems  with  this  class  of  operators  present 
themselves;  both  direct  consequences  of  the  number  of  coefficients  in  the  link  matrix.  Since  each  parent  is  a 
proposition  that  may  assume  2  values,  there  are  2"  combinations  of  possible  values  for  the  set  of  n  parents. 
For  each  matrix,  2”  values  must  be  specified.  Equally  problematic  is  that  once  the  coefficients  have  been 
specified,  the  calculations  required  for  the  evaluation  of  the  child,  Q ,  given  the  probabilities,  p\ .  . . . .  pn ,  for 
the  parents  is  exponential  in  the  number  of  parents.  For  the  general  case,  the  n  term  product: 

n  Pi  n  & 

i(=.R  i£{«i ,  •  -  -  } — R 

must  be  calculated  for  each  of  the  2n  possible  subsets  of  n  parents;  i.e.  each  of  the  2"  possible  combinations 
of  true  parents.  The  vast  universe  of  possible  operators  from  which  to  choose,  coupled  with  the  intractable 
computational  properties  of  these  operators  in  the  general  case  motivates  the  search  for  a  subclass  of  link 
matrices  that: 

•  correspond  to  psychologically  plausible  ways  of  combining  evidence,  and 

•  can  be  put  into  a  tractable  computational  form 

In  his  dissertation,  Howard  Turtle  refers  to  link  matrix  classes  with  the  above  two  properties  as  canonical 
forms.  He  presents  four  such  canonical  forms;  each  general  in  the  sense  that  there  is  an  operator  (or  set  of 
operators)  for  any  number  of  parents.  Here  we  see  the  form  of  these  matrices  for  the  case  of  n=3  parents: 


L and • 


These  canonical  forms  were  motivated  by  operators  traditionally  found  in  IR  systems.  L an(j  and  Lor  can  be 
considered  probabilistic  versions  of  the  AND  and  OR  boolean  operators  offered  in  boolean  retrieval  systems. 
It  can  easily  be  shown  that,  given  probabilities,  pi, . . . . pn ,  for  the  n  constituent  queries  being  satisfied,  the 
probability  that  the  composite  query  is  satisfied  can  be  calculated  simply  as: 

pr(Q  is  true)  =  pip2---p„ 


for  Lanch  and 


pr(Q  is  true)  =  1  -  (1  -  pi)(l  -  p2) . . .  (1  -  p„) 


for  L  or- 
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The  coefficient  for  a  column  of  the  general  n-ary  L sum  matrix  for  which  i  of  the  parents  are  true  is,  A  If 
the  probabilities  are  viewed  as  weights  of  evidence,  the  matrix  can  be  viewed  as  an  operator  which  averages 
these  weights.  That  is: 

PrWis»ue)  =  Pl+R  +  -+P" 
n 

The  L weighted-surn  matr'x  is  a  generalization  of  the  L sum  matrix  in  that  a  weighted  average  is  calculated 
with,  Wi, . . . ,  wn  being  the,  fixed  but  arbitrarily  chosen,  weights  of  the  n  parents,  and  wq  a  weight  associated 
with  the  child  node  which  is  applied  to  the  resultant  average. 

pr(Q  is  true)  =  (wipi  +  W2P2  +  •••  +  wnpn)t 

U’lpi  +  W2P2  +  -"  +  U’npn  ■„ 

W1+W2  +  ...+U>« 


1.3  pic  matrices 

One  interesting  class  of  link  matrices  worthy  of  study  are  those,  such  as  the  L sum  matrix  discussed  above, 
that  do  not  distinguish  among  the  parents.  In  these  matrices,  the  coefficient  in  each  column  is  a  function  only 
of  the  number  of  parents  that  are  true  for  the  corresponding  event,  independent  of  which  parents  they  happen 
to  be.  After  introducing  notational  conventions  that  will  be  used  throughout  this  discussion,  a  more  formal 
definition  of  this  class  of  matrices  is  presented. 

Notation:  In  this  paper  we  will  concentrate  on  the  evaluation  of  one  particular  link  matrix  which  we  shall 
refer  to  as  Lq.  We  will  consistently  use  n  for  the  number  of  parent  nodes  associated  with  this  matrix; 
Pi, Pn  to  represent  the  propositions  corresponding  to  these  parents  nodes;  and  Q  to  represent  the  propo¬ 
sition  corresponding  to  the  child  node. 

Notation:  We  shall  use  pi  to  denote  the  probability  that  the  ith  parent  is  true  and  pi  to  denote  the  probability 
that  it  is  false: 

Pi  =  pr(Pj  is  true) 

Pi  =  pr{Pi  is  false)  =  (1  -  pf) 


Notation:  Given  an  arbitrary  subset,  R ,  of  {1, . . . ,  n},  aR  shall  denote  the  link  matrix  coefficient  corre¬ 
sponding  to  the  conditional  probability  that  Q  is  true  given  that  the  parents.  Pi  such  that  i  €  R,  are  true  and 
the  other  parents  are  false: 

OiR  =  pr( Q  is  true  |  Vi  =  1, . . . ,  n  :  i  €  R  ^  Pi  is  true, 

i  ^  R  ^  Pi  is  false) 


Definition:  When  the  conditional  probability  that  Q  is  true  is  a  function  only  of  the  number  of  parents  that 
are  true  we  shall  say  that  the  parent  indifference  criterion  is  met,  or  simply  that  the  matrix  is  a  pic  matrix. 
Equivalently, 

VR1,R2  C  {1, . . . ,  n}  :  |i?i |  =  |P2|  =>  or  1  =  or2 


Notation:  When  a  matrix  satisfies  the  parent  indifference  criterion,  ctj  shall  be  used  to  denote  the  conditional 
probability  that  Q  is  true  given  that  exactly  j  of  the  parents  are  true. 

°ij  =  pr(Q  is  true  |  |{0  <  i  <  n|P,  is  true}|  =  j) 
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The  sequence  of  link  matrix  coefficients,  ao,  ■  ■  ■ ,  cnn,  can  be  viewed  as  a  function  from  the  integers  {0, 1, . . . ,  n} 
to  the  interval  [0,1],  It  is  appropriate  to  note  that  we  have  in  mind,  and  will  be  exploring,  coefficients  corre¬ 
sponding  to  non-decreasing  functions.  Viewing  the  pic  matrices  as  operators  for  the  combination  of  evidence, 
our  interest  is  in  those  operators  for  which  more  pieces  of  individual  evidence  (i.e.  greater  number  of  true 
parents)  translates  to  greater  probability  that  the  combined  evidence  is  present.  Nonetheless,  the  formal  de¬ 
velopment  to  follow  does  not  depend  on  this  property  of  the  functions,  and  so  it  has  not  been  incorporated  as 
part  of  the  definition  of  pic  matrices. 


1.4  this  study 

Howard  Turtle  has  suggested  further  study  of  matrices  which  satisfy  what  I  have  called  the  parent  indifference 
criterion,  which  has  been  the  initial  impulse  behind  the  work  discussed  here.  It  should  be  noted,  that  the 
L and  and  Lor  matrices  meet  the  parent  indifference  criterion.  As  such,  they  are  special  cases  of  the  class  of 
matrices  under  study.  A  satisfactory  approach  to  computing  with  PIC  matrices  will  allow  for  the  definition 
of  less  stringent  versions  of  these  operators,  which  may  be.  Turtle  speculates,  more  in  accord  with  users’ 
intuitive  understanding  of  what  they  are  trying  to  express  when  they  utilize  them. 

In  what  follows,  I  will  present  an  algorithm  for  evaluating  an  arbitrary  pic  matrix  which  requires  0(n2) 
operations.  Later  I  define  a  more  general  class  of  matrices  for  which  the  parent  indifference  criterion  is 
relaxed  to  allow  for  a  weighting  of  the  parents.  A  simple  modification  of  the  basic  algorithm  expands  its 
applicability  to  include  this  more  general  class. 

Again  following  Howard  Turtle’s  suggestions,  I  also  explore  specializations  of  the  pic  matrix  class.  Viewing 
the  set  of  coefficients  of  a  pic  matrix  as  a  function  on  the  number  of  parents  that  are  true,  we  take  a  look  at 
link  matrices  associated  with  piecewise  linear  functions.  We  see  that  these  may  be  evaluated  in  0(n )  time  in 
certain  restricted  cases. 


2  Basic  Algorithm 

In  this  section,  an  algorithm,  pic-eval,  is  presented  for  evaluating  link  matrices  that  meet  the  parent  indiffer¬ 
ence  criterion.  Starting  with  the  original  link  matrix,  L o,  pic-eval,  in  effect,  generates  a  sequence  of  smaller 
and  smaller  link  matrices,  L±,  L2,  ■  ■  ■  ,Ln.  In  the  process,  it  eliminates  from  consideration  each  probability 
in  turn  (Figure  3). 


2.1  description  of  the  pic-eval  algorithm 

To  begin,  the  probability  associated  with  parent  node.  Pi ,  is  fixed  and  the  matrix,  Lo,  with  n  +  1  coefficients 
and  n  parents  (figure  3a)  is  converted  to  an  n  coefficient  link  matrix  with  n  —  1  parents  (figure  3b).  The 
matrix,  L\,  that  results  from  this  transformation  is  equivalent  to  the  original  matrix,  L 0,  in  the  following 
sense: 


for  any  set  of  probabilities  for  parents,  P2,  . . . ,  Pn,  L 1  yields  the  same  pr(  Q  is  true)  that  Lq 
would  produce  given  the  same  probabilities  for  P2,  ....  Pn,  together  with  the  probability  pi  for 

Pi. 

Notation:  With  respect  to  the  execution  of  the  pic  matrix  evaluation  algorithm:  Li  shall  refer  to  the  link 
matrix  generated  during  the  iteration,  and  the  coefficient  of  L  -t  associated  with  j  parents  being  true  shall 
be  referred  to  as  a(i ,  j).  We  note  that  for  the  initial  matrix,  Lo : 

a(0,j)  =  aj  Vi  =  0,  ...,n 
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Figure  3:  iterations  in  the  evaluation  of  a  pic  matrix 


inputs: 

•  n  +  1  coefficients,  Qo,  ■  ■ . an,  of  a  pic  matrix 

•  probabilities  Pi,  ■  ■  ■  ,pn,  for  the  n  parent  nodes, 
output: 

•  the  probability  that  the  child  node  is  true. 

pic-eval: 

for  j  =  0, ...  ,n 
a[0  ,j]  =  Oij 

for i  =  1 ..... n 

for  j  =  0  —  i 

a[i,j]  «-  a[i  -  1  ,j]  *  (1  -  pt)  +  a[i  -  1  ,j  +  1  ]*pt 
return  a\n,  0] 


Figure  4:  pic-eval  algorithm 
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As  shown  in  figure  3,  each  matrix  has  one  coefficient  fewer  than  the  previous  one  and  connects  with  one  fewer 
parent  node.  After  n  iterations  we  arrive  at  Ln  with  exactly  one  coefficient,  a(n,  0),  as  seen  in  figure  3e. 
Given  that  all  parent  probabilities  have  been  accounted  for,  this  coefficient  may  be  interpreted  as  the  desired 
probability  that  the  child  is  true.  This  general  strategy  is  embodied  in  the  pic-eval  algorithm  given  in  figure  4. 


2.2  correctness  of  pic-eval  algorithm 

The  correctness  of  the  pic  algorithm  follows  directly  from  Lemma  1,  which  is  more  easily  expressed  by 
adopting  the  following  notation. 

Notation:  Given  an  arbitrary  subset,  R,  of  {ii# *  . . ,  },  shall  denote  the  probability  that  precisely 

the  parents.  Pi  such  that  i  €  R,  are  true,  while  those  parents.  Pi  such  that  i  (j  R,  are  false.  Equivalently: 

4— >=n^  n  p* 

i(=.R  R 


Notation:  The  notation  will  be  used  to  denote  the  probability  that  exactly  j  of  the  propositions 

of  {Pi,,...,  Pi2 }  are  true.  We  observe  that: 


T{*1.  — .»2>  _ 


=  E 


1,  — ,*2> 


\R\=j 


It  is  worth  noting  that  the  probability  that  Q  is  true  can  be  given  by: 

pr(Q  is  true)  =  E  [«jt  n  Pi  n 

sc{i . r,}  ten 

=  Y 

HC{1, 

And  that  when  the  parent  indifference  criterion  is  met: 


pr(Q  is  true)  =  ^  [ort r^1,‘”,n}]  =  ^  ^  [^R^r 


3  =  0  BC{1 n} 

|H|=j 


3=0  HC{l,...,n} 
I R I  =3 


=  Y^  Y 


7T 


R  J 


3=0  flC{l,...,n} 

\R\=j 


=  E 


OtjCJj 


3-0 


Lemma  1  Assuming  all  coefficients  a(i,  j)  are  those  produced  by  the  pic-eval  algorithm,  then  Vi  =  0, 1, . . . ,  n 


Ya^^ 

3=0 


a{i+ 1. 
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pr(Q  is  true) 


Lemma  1  effectively  states  that  the  set  of  coefficients,  a(i,  0), . . . ,  a(i,  n  —  i),  can  be  interpreted  as  specify¬ 
ing  a  pic  matrix,  Li,  connecting  parent  nodes  Pi+i ,  . . . ,  Pn,  to  Q,  and  that  this  matrix  is,  in  a  sense,  equivalent 
to  the  original  matrix,  Lq. 

proof:  This  lemma  is  proved  by  induction  on  the  value  of  i.  For  i  =  0,  we  have: 


yy,i'o:  jirrj1 


X>  . 

3=0 

\R\=j 

e  E^n«n«] 

j=o  ieR  igR 

I R I  =3 

e  \-aR  n^n 

«C{l,.,n]  iER  i£R 

pr(Q  is  true) 


Assume  the  theorem  to  be  true  for  i  =  k: 

n  —  k 

y^q(fc,  =  pr(Q  is  true) 

i= 0 


Then  the  sum  for  i  =  k  +  1  is: 

n-k-1 

E  a{k  +  l,j)a. 


3 


=  E  ia(kJ)Pk+ 1  +  ai(k,j  +  l)pk+1\a 


{k+2,...,n} 

3 


3=0 


(by  step  (5)  of  the  algorithm) 

i—k—l  n-k-1 

{k+2 
7 3 

j  =0  j  =0 

i-fe-1  n—k 

{k+2 


=  e  a(kij)pk+i°j 


+  *22  a(k,j  +  l)pk+1cr 


{k+2 

3 


{k+2,...,n} 


^2  a(k’j)Pk+l<7j2'm"'ni  +  fipk+lOj+i 

3=0  3=1 

(as  a  result  of  changing  the  variable  for  the  second  summation) 

u(k,0)pk+ivok+2’"'’n} 


{k+2,...,n} 


{k+2,...,n} 


The  term  <r( 

composed  of  only  one  product.  Hence,  the  first  term  of  ( 1)  reduces  to: 

a(k,0)pk+1a^k+2’"',n}  ~ 


(1) 


+  E  a(k,j)pk+i<Jj’"',n}+  E  a(k>  fiPk+iVj-t 

j= 1  3= 1 

+a(k,n  -  k)pk+la{2^2-Tn} 

,  expresses  the  probability  that  none  of  the  propositions,  Pk+ 2, . . . ,  Pn,  is  true.  It  is 


=  a(k,0)pk+1  pi 

l=k+ 2 
n 

=  a(k,  0)  pi 
i=k+ 1 

=  a(k,0)aQk+1,'"’n^ 


(2) 


9 


Similarly,  cr^^+i  ’  which  expresses  the  probability  that  all  n  —  k  +  1  of  the  propositions  { Pk+2 , . . . ,  P„} 
are  true,  is  composed  of  only  one  product.  Therefore,  the  last  term  of  (  1)  reduces  to: 

n 

a(k,n-  A0Pfc+i^-l+r"}  =  a{k,n-  k)pk+1  pt 

l=k+ 2 
n 

=  a(k,  n  —  k)  pi 


l=k+ 1 

=  a(k,n-  k)a^-",n} 


Combining  the  middle  two  terms  of  (  1): 


Y  a(k,j)pk+la 


{fc+2,...,n} 


+  Y  a(k’j)Pk+lVj-t 


{fc+2,...,n} 


Y  a(k’j)[(Pk+iajk+2'"'’n})  +  iPk+iofX2,'"’n})] 


3  =  1 
i-k-1 


=  Y  a(k’j)i(pk+ 1  Y 


7T 


{fc+2,...,n} 
i? 


)  +  (p*+t  Y 


7T 


{fe+2,...,n} 


)] 


RC{fc+2,...,w} 

|H|=J 


RC{fc+2,-,n} 
I  -R I  =3  —  i 


=  ^2<*(k,j)[(  Pk+lKR""'n})  +  (  Pfc+1 


{fc+2,...,n} 

lR 


)] 


RC{fc  +  l,...,n> 
|H|=i 


BC{fc+l . »} 


=  y  a(*>j)[  y 


lR 


TTb  '  '  J  +  ^  7T 


i?  J 


RC{fc  +  l, 
|R|=jAfc  +  lgR 


BC{fc  +  l »} 

|R|=jAfe+l€H 


=  y  y 


7T 


i?  J 


j=i 

-t-fc-i 


RC{fe  +  l,...,r»} 
I  -R I  =3 


{k+l,...,n} 
3 


=  ^2  a(k>3)aj 

3  =  1 

Finally,  applying  equations  (  2),  (  3),  and  (  4)  to  equation  (  1),  we  have: 

!>(*+ i,j>j*+2 . 

j=0 

=  a(fc,  0)CT(5fc+1’"'’"}  +  [  Y  a(fc’  +  a(A,  n  - 


(3) 


(4) 


=  ya(fc,j)<7 

3=0 

=  pr(<5  is  true) 


•  n  {*+!,. ..,n} 


(by  the  induction  hypothesis) 


Setting  i  =  n  in  Lemma  1  immediately  yields  the  following  theorem  which  states  that  the  pic-eval  algorithm 
is  correct. 


Theorem  1  Given  that  a(n,  0)  has  been  produced  by  the  pic-eval  algorithm: 

a{n,  0)  =  pr(Q  is  true ) 


10 


2.3  efficiency  of  the  pic-eval  algorithm 

The  pic-eval  algorithm  is  clearly  executed  in  0(n2)  time.  The  initialization  requires  0 (n)  and  the  main  loop 
is  executed  precisely 


n  n—i  n 


E£i=I>- 


n+1 

i  +  1)  =  ^  i  =  (n  +  2  )(n  +  l)/2  =  0(n2) 
i=  1 


times.  Also,  the  constant  factor  is  small.  On  top  of  the  base  iteration  control  overhead,  two  multiplications 
and  one  addition  are  required  in  each  iteration. 

Although,  for  the  purposes  of  exposition,  the  algorithm  has  been  shown  as  requiring  0(n2)  space  as  well  as 
time,  0(n )  space  is  easily  achieved  since  only  one  row’s  worth  of  cells  need  be  maintained  at  any  one  time. 


3  Piecewise  Linear  Functions 

In  this  section  we  will  look  at  certain  PIC  matrices  for  which  the  evaluation  algorithm  can  be  made  more 
efficient. 

The  pic-eval  algorithm  can  be  viewed  as  filling  a  triangular  portion  of  a  square  matrix  as  shown  in  figure  5a. 
The  first  row  is  initialized  with  the  aj  values  and  then  each  row  from  1  to  n  is  processed  in  turn.  Within  each 
row,  the  cells  are  set  from  left  to  right,  with  each  cell  set  to  the  sum  of: 

•  the  cell  immediately  above  it,  and 

•  the  cell  above  it  and  to  the  right. 

As  a  consequence  of  this,  the  value  of  each  cell,  a(i ,  j),  is  dependent  only  on  the  values  of  cells  in  a  triangle 
extending  directly  above  it  on  the  left  and  at  a  45°  angle  to  the  right,  as  is  shown  in  figure  5b.  That  is,  a(i ,  j ) 
depends  on  only  those  a(k,  l )  such  that: 


0  <  k  <  i 

j  <  l  <  j  +  m  —  i 


When  a  subsequence  of  the  PIC  matrix  coefficients  forms  an  arithmetic  progression, 

Ct/TJ  3  O^rn  T  A,  OLjyi  -}-  2A,  ■  ■  ■  5  OLjyi  T 

the  cell  values  in  the  triangular  subsection  immediately  below  these  coefficients  can  be  expressed  directly  in 
terms  of  am,  A,  and  the  parent  probabilities,  p\ ,  . . . ,  ps .  Of  these  cell  values,  the  only  ones  that  are  needed 
for  calculations  outside  of  the  triangle  are  those  on  the  leftmost  edge: 

a(0,j),...,a(i,j) 

Hence,  if  there  were  a  direct  method  for  determining  these  cell  values,  the  rest  of  the  cells  in  the  triangle 
need  not  be  calculated  at  all.  Exactly,  how  this  may  be  accomplished  follows  from  the  following  Lemma. 


Lemma  1  Given  a  PIC  matrix  whose  coefficients, 

Ctm+1 ,  (Xm- (-2,  ■  ■  ■  ,  CKm+s 


are  of  the  form: 


T  A,  Oim  +  2A,  .  .  .  ,  Oim  -f-  s A 


i.e.,  are  such  that: 

Um+j=Um+j  A  Vj  =  0;...,S 
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Figure  5:  cell  evaluation  during  execution  of  pic-eval 
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then,  Vi  =  0, . . . ,  s  Vj  =  0, . . . ,  s  —  i: 

i 

a(i,  m  +  j)  =am+  j  A  +  A^pj 

1  =  1 

proof:  By  induction  on  i.  The  base  case  follows  directly  from  the  hypothesis  of  the  lemma. 

a(0,m  +  j)  =am+j  =  am  +  j  A 

0 

=  Oim  +  j  A  +  AYjPj 
1=1 

Assuming  the  lemma  to  be  true  for  i=k-l: 
a(k,m  +  j) 

=  a(k-l,m  +  j)(l-pk)  +  a(k-l,m  +  j  +  l)pk 

k— 1  k—1 

=  («™  +jA  +  A^2pi)(1  -pk)  +  (am  +  (j +  1)A  + A^prfpk 

1=1  1=1 

k—1  k-1 

=  am  +  (jA  +  A^]p/)(1  -pk)  +  ((j  +  1)A  +  A^pi)pk 

1=1  1=1 

k-1  k-1 

=  am  +  j  A  +  (A  ^Pi)  -  jApk  -  (A^2pi)pk 

1=1  1=1 

k-1 

+  (j  +  1)A  pk  +  (A  ^2pi)pk 
1  =  1 

k—1 

=  am  +  jA  +  (A ^Pi)  -  jApk  +  (j  +  l)Apfe 

1  =  1 
k-1 

=  am  +  jA  +  (A  ^pi)  +  A  pk 

1  =  1 
k 

=  am  +  jA  +  (A  ^Pi) 

1  =  1 


In  particular,  each  of  the  cells  at  the  left  edge  of  the  triangle  can  be  computed  as: 


a(i,m )  =  am+0A  +  (A^pfc) 

fe  =  l 
i  —  1 

=  am  +  0A  +  (Ayn)  +  A  pi  =  a(i  -  1,  m)pi  +  A  pt 

k  =  l 

which  requires  a  total  of  only  s  additions  and  s  multiplications,  replacing  the  Yli=i  =  s(s  +  l)/2  general 
cell  computations  which  would  normally  be  executed. 

This  technique  can  result  in  substantial  savings  if  the  number  of  coefficients  involved,  and  hence  the  size  of 
the  triangle  involved,  is  large.  In  his  notes,  Howard  Turtle,  has  suggested  looking  at  the  case  where  the  PIC 
matrix  coefficients  correspond  to  a  piecewise  linear  function  on  the  number  of  true  parents.  For  example, 
figure  6a,  shows  a  function  with  three  linear  pieces.  The  matrix  coefficients  in  this  case  comprise  three 
arithmetic  series,  each  associated  with  a  corresponding  savings  in  cost  of  evaluation.  Since  the  domain  of  the 
function  is  discrete,  it  is  not  strictly  necessary  that  the  pieces  connect.  Therefore,  when  speaking  of  piecewise 
linear  functions,  we  shall  also  consider  functions  such  as  that  shown  in  figure  6b. 

The  form  of  the  2-piece  piecewise  linear  functions  shown  in  figures  6c  and  6d  are  of  interest  because  they  can 
be  interpreted  as  generalizations  of  the  L anc\  and  L or  link  matrices.  We  will  see  in  a  moment  that  evaluation 
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a)  3  piece  function 


b)  also  3  piece  function 


c)  generalizatin  of  AND 


d)  generlization  of  OR 


l  - 


Figure  6:  piecewise  linear  functions 
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of  these  functions  is  particularly  efficient.  The  function  shown  in  figure  6c,  for  instance,  generalizes  the  Lanc] 
matrix  in  that  the  conditional  probability  that  Q  is  true  may: 

•  take  on  a  (presumably  small)  value  greater  than  0  when  all  parents  are  false. 

•  rise  (presumably  slowly)  at  a  constant  rate  as  more  parents  are  known  to  be  true. 

•  rise  suddenly  for  some  number  of  true  parents  m  less  than  n  -  although  m  must  be  less  than  n  by  some 
(presumably  small)  constant  value,  independent  of  n. 

•  rise  at  a  constant  rate  as  the  number  of  parents  known  to  be  true  goes  from  m  to  n 

•  take  on  a  value  less  than  1  when  all  parents  are  true. 

In  a  similar  fashion,  the  form  shown  in  figure  6d,  generalizes  the  L or  matrix. 


Figure  7:  evaluation  of  generalized  OR 


Although  the  savings  realized  for  a  piecewise  linear  function  may  be  significant,  the  asymptotic  cost  of 
execution  will  not  be  affected  unless  all  of  the  pieces  save  one  cover  a  constant  number  of  coefficients  2.  If 
am,  &m+u  •  •  ■ ,  as ,  is  an  arithmetic  series,  then  s(s  +  l)/2  cell  operations  can  be  eliminated  in  favor  of  s 
additions.  This  leaves. 


n(n  +  l)/2  —  s(s  +  l)/2 


(n2  +  n  -  s2  -  s)/ 2 

ns  —  s2  +  (n2  —  ns  +  n  +  s2  —  ns  —  s)  /  2 
(n  —  s)s  +  (n(n  —  s  +  1)  —  s(n  —  s  +  l)/2 
(n  —  s)s  +  (n  —  s)(n  —  s  +  l)/2 


When  (n  —  s )  is  constant,  the  second  term  is  constant  and  the  first  term  grows  with  s.  Since  s  must  grow  with 
n,  if  n  —  s  is  to  be  kept  constant,  the  cost  of  evaluation  is  0(n).  Figure  7  shows,  pictorially,  how  the  array 
for  the  generalized  OR  function  is  evaluated.  The  cells  which  are  evaluated  are  restricted  to  a  fixed  width 
strip  at  the  left  of  the  array  whose  length  grows  with  n. 


4  Weighted  Parents 

In  the  previous  section  we  defined,  and  developed  an  0(n2)  algorithm  for,  the  class  of  link  matrices  for  which 
the  conditional  probabilities  are  dependent  only  on  the  number  of  parents  that  are  true.  In  this  section,  we 
generalize  this  result,  beginning  with  the  class  of  link  matrices  under  consideration. 

The  goal  of  this  generalization  is  to  allow  for  the  possibility  that  the  truth  of  each  proposition,  Pj,  may  have  a 
different  impact  on  the  probability,  pr(Q  is  true).  There  will  always  be  one  parent  whose  impact  is  at  least  as 
great  as  any  other.  For  the  purposes  of  this  exposition  we  will  assume,  without  loss  of  generality,  that  parent. 
Pi,  has  this  property.  The  impact  of  each  other  parent  may,  then,  be  weighted  by  some  factor,  0  <  Wi  <  1.0. 
For  generality,  we  will  say  that  all  parents  are  weighted  and  that  the  weight,  Wi ,  of  parent  Pi  is  equal  to  1 . 
We  shall  say  that  a  link  matrix,  satisfies  the  weighted-parent  indifference  criterion  when  the  probability  that 

-Prof.  David  Barrington  has  pointed  out  that  computational  improvement  may  be  realizable  even  if  this  condition  is  not  fully  met. 
For  example,  if  one  piece  grows  with  n,  while  all  others  grow  only  as  login),  the  asymptotic  running  time  will  be  only  O{nlog{n)). 
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the  child  is  true  is  strictly  a  function  of  the  number  of  parents  that  are  true,  once  the  weights  of  the  parents 
have  been  taken  into  consideration.  Formally, 

Definition:  We  shall  say  that  the  weighted-parent  indifference  criterion  is  met  when: 

3  (wi  =  1.0, 

0  <  w2,  ■  ■  ■  ,wn  <  1.0 
0  <  Q0,...,Q„  <  1.0) 

VHC{l,...,n}  cur  =  otj  Wi 

i£R 


inputs: 

•  n  +  1  coefficients,  Qo, . . . ,  a„,  of  a  pic  matrix 

•  probabilities  pi , . . . ,  p„,  for  the  n  parent  nodes, 
output: 

•  the  probability  that  the  child  node  is  true. 

pic-eval: 

for  j  n 

<*[0  ,j]  =  aj 

for  i  =  1, . . , ,  n 

for  j  =  0, . n  —  i 

a[i,j]  +-  a[i  -  1  ,j]  *  (1  -  pi)  +  a[i  -  l,j  +  1  ]*wt*Pi 
return  a\n,  0] 


Figure  8:  wpic-eval  algorithm 

We  shall  refer  to  a  link  matrix  satisfying  the  weighted-parent  indifference  criterion  as  a  WPIC  matrix.  A  WPIC 
matrix  is  completely  determined  when  two  sets  of  parameters  have  been  given:  Qo,  ■  ■  ■ ,  otn  and  UJ\ . ■  wn. 
When  all  the  parents  have  a  weight  of,  =  1,  the  WPIC  matrix  reduces  to  a  simple  PIC  matrix,  where  the 
coefficient,  a 3 ,  specifies  the  probability  that  Q  is  true  when  it  is  known  that  j  parents  are  true. 

For  the  general  WPIC  matrix,  however,  rather  than  specify  what  the  probability  that  Q  is  true  is,  for  j  true 
parents,  the  coefficient,  aj,  specifies  what  that  probability  would  be  for  j  true  parents,  if  all  the  parents  had 
the  same  impact;  that  is,  if  all  the  weights  were  1.  A  weight,  Wi  <  1,  indicates  that  when  is  one  of  the  true 
parents,  the  probability,  pr(Q  is  true),  is  less  than  it  would  be  were  P,  to  have  the  same  weight  as  Pi.  The 
weight,  Wi,  gives  the  factor  by  which  pr( Q  is  true)  must  be  discounted  when  P;,  rather  than  a  parent  whose 
impact  is  equal  to  that  of  Pi ,  is  one  of  the  true  parents  3. 

The  basic  algorithm  requires  only  a  minor  modification  in  order  that  WPIC  matrices  be  processed  correctly. 
The  wpic-eval  algorithm  incorporating  the  necessary  modification  is  given  in  figure  ??. 

Lemma  1  and  Theorem  1,  used  to  demonstrate  the  correctness  of  the  pic-eval  algorithm,  can  be  modified  to 
contemplate  weights.  The  reasoning  used  for  their  proofs  applies  with  very  minor  modifications.  Appendix  A, 
presents  generalized  versions  of  the  notation  introduced  for  PIC  matrices  and  a  modified  version  of  Lemma  1. 
For  reference,  the  complete  proof  for  the  new  version  of  the  lemma  is  included,  although  it  is  essentially  the 
same  as  that  given  for  Lemma  1  in  section  2. 

^Another  way  of  understanding  the  WPIC  matrices  is  in  terms  of  gated  inputs.  Each  parent  is  viewed  as  one  input  to  an  AND  gate 
whose  other  input  corresponds  to  an  independent  activation  variable.  Only  when  both  the  parent  and  the  activation  variable  are  both 
true  is  a  value  of  true  seen  by  the  Q  node.  The  parent  weight  becomes  the  probability  that  the  activation  variable  is  true.  From  this, 
we  see  that  the  class  of  WPIC  matrices  can  be  viewed  as  a  generalization  of  the  noisy  or  matrices  often  utilized  in  Bayesian  Network 
applications. 
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It  should  be  observed  here  that,  with  respect  to  computational  efficiency,  the  only  difference  between  the 
two  versions  of  the  algorithms  is  that  the  weighted  algorithm  requires  an  extra  multiplication  during  each 
execution  of  the  main  loop. 


5  Summary 

In  this  paper  I  have  described: 

•  the  PIC  matrices,  a  subclass  of  link  matrices  which  would  appear  to  correspond  to  a  natural  class  of  retrieval 
operators. 

•  an  0(n2)  algorithm,  pic-eval,  for  the  evaluation  of  arbitrary  PIC  matrices. 

•  an  improvement  to  the  general  pic-eval  algorithm  when  a  subsequence  of  the  link  matrix  coefficients  form 
an  arithmetic  series.  This  improvement  results  in  an  0(n)  algorithm  for  certain  forms  of  link  matrices 
which  may  be  considered  generalizations  of  Turtle's  L an(j  and  Lor  matrices. 

•  the  WPIC  matrices,  which  allow  for  the  weighting  of  parent  nodes  and  a  simple  modification  to  the  basic 
algorithm  which  adjusts  for  this  generalization. 

In  closing,  some  aspects  of  the  link  matrix  evaluation  problem  intimately  related  to  the  ideas  presented  here, 
can  be  mentioned. 

Piecewise  linear  functions  may  be  evaluated  more  efficiently,  in  some  cases  much  more  efficiently,  than  an 
arbitrary  PIC  matrix.  Unfortunately  it  does  not  immediately  follow  that  these  gains  in  efficiency  may  be 
realized  when  the  parents  are  weighted.  A  closer  look  at  this  problem  may  yield  an  interesting  subclass  of 
the  WPIC  matrices  that  may  also  be  evaluated  more  efficiently. 

When  a  subsequence  of  the  PIC  matrix  coefficients  form  an  arithmetic  series,  the  need  for  a  full  evaluation 
of  the  triangle  underneath  the  subsequence  is  eliminated.  It  may  be  fruitful  to  investigate  other  properties 
that  might  result  in  similar  shortcuts  to  the  computation  of  the  entire  0(n2)  matrix  that  is  required  for  the 
evaluation  of  an  arbitrary  PIC  matrix. 

In  a  recent  discussion  with  Prof.  Eliot  Moss,  he  suggested  to  me  an  interesting  approach  to  the  efficient 
evaluation  of  WPIC  matrices.  He  suggests  using 


j  =  ^2 wiPi 

i  =  1 

as  an  estimate  of  the  number  of  parents  that  are  true.  An  arbitrary  function  on  the  real  interval  [0, 1]  now 
takes  the  place  of  the  link  matrix.  This  function  is  evaluated  at  j  to  yield  an  approximation  of  the  probability, 
pr(Q  is  true).  While  this  technique  is  not  equivalent  to  the  specification  of  a  link  matrix,  under  appropriate 
conditions  it  might  provide  a  practical  alternative. 
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7  Appendix  A:  Correctness  of  WPIC-EVAL 

Notation:  For  the  purposes  of  the  analysis  of  wpic-eval,  we  will  adopt  the  following  notation  which 
modifies  that  used  previously  for  the  analysis  of  the  pic-eval  version. 

= n m  n  pi 

i(=.R  «G{«i  ,...,*2}- r 

1  )  •  *  *5^2  }  \  A 

ai  =  2^  nR 

\R\=j 


Lemma  1  Assuming  all  coefficients  a(i,j)  are  those  produced  by  the  wpic-eval  algorithm,  we  have:  Vi  = 

0,1 

n  —  i 

j)Ojl+1,'"’n^  =  pr(Q  is  true) 

i-  0 

proof:  This  lemma  is  proved  by  induction  on  the  value  of  i.  For  i  =  0,  we  have: 

=  Eqj  E41’-’"1 

j= 0  j  =0  it  C  { 1 , ,T1 } 

\R\=j 

=  e  e  [qi  n Wipi  n 

j=  0  itc{i . »»}  ieR  i&R 

I R I  =3 

=  e  EKn^nftn^ 

j=o  rc { 1, . . . ,w }  iG-R  iG-R 
|H|=J 

=  E  EtQ«  lift  lift] 

j=  0  RC{1,  ...,n}  «Gi?  i£R 
\R\=3 

=  e  n  ft  n  *] 

=  pr(Q  is  true) 

Assume  the  theorem  to  be  true  for  i  =  k: 

n  —  k 

= Fr(<3 is  true) 

i-o 

Then  the  sum  for  i  =  ft  +  1  is: 

'£'<.(»+ . ”> 

j=0 

w-fe-1 

=  E  [Q(*> +  a(kJ  +  l)wk+iPk+\\(y]k+2’"',n] 

3=0 

(by  step  (5)  of  the  algorithm) 

=  E  a(k,j)pk+iof+2,'",n}  +  E  a(k>j  +  l)wk+ipk+iaf+2,'",n} 

3  =0  j  =0 
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=  Y  a(k,j)Pk+i<7jk+2’"',n}  +  'Y^a{k,j)wk+1pk+1u]k+l2,''',n} 

3=0  j =1 

(as  a  result  of  changing  the  variable  for  the  second  summation) 

=  a(fc,0)^+1<70{ft+2’-’”}  (5) 

+  Y  a(k,j)pk+1a{jk+2,'"’n}+  Y  «(*!j)^*+iP*+io-j-T2,'"’"} 

3= 1  3=  1 

+a(fc,  n  -  k)wk+1pk+1a^+2-rn} 

The  term  cr('|/'+2;'":"-'\  expresses  the  probability  that  none  of  the  propositions,  Pk+ 2, . . . ,  Pn,  is  true.  It  is 
composed  of  only  one  product.  Hence,  the  first  term  of  (  ??)  reduces  to: 

n 

u{k,0)pk+iaok+2’"'’n}  =  «(M)Pfc+t  Pi 

l=k+ 2 
n 

=  a(k,  0)  pi 
l=k+ 1 

=  a(fc,  0)aofc+1’'''’"-^  (6) 

Similarly,  which  expresses  the  probability  that  all  n  —  k  +  1  of  the  propositions  {Pk+ 2,  ■  ■  ■ ,  Pn  } 

are  true,  is  composed  of  only  one  product.  Therefore,  the  last  term  of  (  ??)  reduces  to: 

n 

a(k,n  -  k)wk+ipk+io{2+2+i,n}  =  a(k,  n  -  k)wk+ipk+i  wtpi 

l=k+2 

n 

=  a(k,  n-k )  wipt 

l=k+ 1 

=  a{k,n-k)o{2+2’'’n]  (7) 

Combining  the  middle  two  terms  of  (  ??): 


^ 2  <x(k,j)pk+1ajk+2’"',n}  +  Y  a{k,j)wk+1pk+1af_X2'""n} 

3  = 1  3= 1 

=  Y  a(kJ)[(Pk+iVjk+2’"'’n})  +  {wk+iPk+iVj-T’'"’^)] 


3=  1 
i  —  k—1 


=  ^2  oi(k,j)[{pk+ 1  ^  7rjjfc+2’-’"})  +  (wfc+ipfc+i  Y 


{h+2 


)] 


RC{fc+2, 

\R\=3 


RC{fc+2, 

I R I  =  j  — 1 


j  =  l  flC{fc+l,..,«} 

|R|=j 


-R 


})  +  (  Y  wk+lPk+l^R 


RC{fe  +  l,...,r»} 

I R I  =  j  — 1 


R 


n  '  '  ■  +  y  7T 


i?  J 


«C{fc+l . n} 

|R|=jAfc+lgR 


«C{fc  +  l n} 

I R I  =3  A  fe + 1  €  R 


=  Y  ia(k;j)  Y 


‘.R  J 


BC{fc+l . n} 

I R I  =  j 


j 


(8) 
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Finally,  applying  equations  (  ??),  (  ??),  and  (  ??)  to  equation  (  ??),  we  have: 

■£’o(*  +  l,j)<rj‘+a . ”l 

3=0 

=  a(k,  0)a^k+h'"’n}  +  [  ^2  a(k ’  +  a(k,  n  ~ 

3=  1 

=  ^a(fc,j)CTj*+1’-"’n} 

j=o 

=  pr(Q  is  true)  (by  the  induction  hypothesis) 
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